forked from restic/restic
-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[pull] master from restic:master #201
Open
pull
wants to merge
1,160
commits into
octoleo:master
Choose a base branch
from
restic:master
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This does not produce exactly the same messages, as it inserts newlines instead of "; ". But given how long our error messages can be, that might be a good thing.
…/x/sys-0.25.0 build(deps): bump golang.org/x/sys from 0.24.0 to 0.25.0
…g/automaxprocs-1.6.0 build(deps): bump go.uber.org/automaxprocs from 1.5.3 to 1.6.0
internal/restic: Use IDSet.Clone + use maps package
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.22.0 to 0.23.0. - [Commits](golang/oauth2@v0.22.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/oauth2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>
tag: fix swallowed error if repository cannot be opened
Changelogs should omit problem if its description duplicates the new behavior
generate: allow passing `-` for stdout output
…/minio/minio-go/v7-7.0.77 build(deps): bump github.com/minio/minio-go/v7 from 7.0.76 to 7.0.77
…/x/oauth2-0.23.0 build(deps): bump golang.org/x/oauth2 from 0.22.0 to 0.23.0
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.195.0 to 0.199.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](googleapis/google-api-go-client@v0.195.0...v0.199.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>
…ang.org/api-0.199.0 build(deps): bump google.golang.org/api from 0.195.0 to 0.199.0
Some error handling patches
dump: Simplify writeNode and use fewer goroutines
…ndows fs: fix restoring timestamps on older Windows versions for long paths
restic cannot backup irregular files as those don't behave like normal files. Thus skip them with an error.
backup: fix handling of files with type irregular
dump: add --compress flag to compress archives
fs: move NodeFromFileInfo into FS interface
Ignore AuthorizationFailure caused by using a container level SAS/SAT token when calling GetProperties during the Create() call. This is because the GetProperties call expects an Account Level token, and the container level token simply lacks the appropriate permissions. Supressing the Authorization Failure is OK, because if the token is actually invalid, this is caught elsewhere when we try to actually use the token to do work.
Add changelog entry in the 'unreleased' sub-folder for changes introduced when fixing issue #4004.
backup: allow excluding online-only cloud files
Stabilize `explicit-s3-anonymous-auth` and `safe-forget-keep-tags` feature flags
This fixes an issue where restic cannot find the tree when trying to find the tree id of a snapshot. --------- Co-authored-by: Albin Vass <[email protected]> Co-authored-by: Michael Eischer <[email protected]>
Those methods now only allow modifying snapshots. Internal data types used by the repository are now read-only. The repository-internal code can bypass the restrictions by wrapping the repository in an `internalRepository` type. The restriction itself is implemented by using a new datatype WriteableFileType in the SaveUnpacked and RemoveUnpacked methods. This statically ensures that code cannot bypass the access restrictions. The test changes are somewhat noisy as some of them modify repository internals and therefore require some way to bypass the access restrictions. This works by capturing an `internalRepository` or `Backend` when creating the Repository using a test helper function.
…cked repository: restrict SaveUnpacked and RemoveUnpacked
* tag: output the original ID and new snapshotID tag: print changed snapshot information immediately * print changed snapshot immediately after it has been saved * add message type to the changedSnapshot * add a summary type which will share the JSON output of the numer of changed snapshots * updated verbosity of the changed snapshot in text mode to only work when verbosity > 2 * also use the terminal status printer for a standard handling for stdout messages
…tore Allow excluding xattrs at restore time
Bump dockerfile to go 1.23
The size comparison for `--max-unused` only accounted for unused but not for duplicate data. For repositories with a large amount of duplicates this can result in a situation where no data gets pruned even though the amount of unused data is much higher than specified.
…plicates prune: correctly account for duplicates in max-unused check
Removing the shadowing cases leads to weird workarounds but doesn't help much with code clarity.
CI: bump golangci-lint and silence warnings about shadowed builtins
…5184) The old sorting behaviour was to sort snapshots from oldest to newest. The new sorting order is from newest to oldest. If one wants to revert to the old behaviour, use the option --reverse. --------- Co-authored-by: Michael Eischer <[email protected]>
Bumps [github.com/spf13/pflag](https://github.com/spf13/pflag) from 1.0.5 to 1.0.6. - [Release notes](https://github.com/spf13/pflag/releases) - [Commits](spf13/pflag@v1.0.5...v1.0.6) --- updated-dependencies: - dependency-name: github.com/spf13/pflag dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go) from 1.8.0 to 1.8.1. - [Release notes](https://github.com/Azure/azure-sdk-for-go/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md) - [Commits](Azure/azure-sdk-for-go@sdk/azcore/v1.8.0...sdk/azidentity/v1.8.1) --- updated-dependencies: - dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>
…/Azure/azure-sdk-for-go/sdk/azidentity-1.8.1 build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azidentity from 1.8.0 to 1.8.1
Bumps [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go) from 1.5.0 to 1.6.0. - [Release notes](https://github.com/Azure/azure-sdk-for-go/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md) - [Commits](Azure/azure-sdk-for-go@sdk/azcore/v1.5.0...sdk/azcore/v1.6.0) --- updated-dependencies: - dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>
…/spf13/pflag-1.0.6 build(deps): bump github.com/spf13/pflag from 1.0.5 to 1.0.6
ui/termstatus: Remove unused bytes.Buffer
…/Azure/azure-sdk-for-go/sdk/storage/azblob-1.6.0 build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 1.5.0 to 1.6.0
doc: add edit on github link
) * feat(backends/s3): add warmup support before repacks and restores This commit introduces basic support for transitioning pack files stored in cold storage to hot storage on S3 and S3-compatible providers. To prevent unexpected behavior for existing users, the feature is gated behind new flags: - `s3.enable-restore`: opt-in flag (defaults to false) - `s3.restore-days`: number of days for the restored objects to remain in hot storage (defaults to `7`) - `s3.restore-timeout`: maximum time to wait for a single restoration (default to `1 day`) - `s3.restore-tier`: retrieval tier at which the restore will be processed. (default to `Standard`) As restoration times can be lengthy, this implementation preemptively restores selected packs to prevent incessant restore-delays during downloads. This is slightly sub-optimal as we could process packs out-of-order (as soon as they're transitioned), but this would really add too much complexity for a marginal gain in speed. To maintain simplicity and prevent resources exhautions with lots of packs, no new concurrency mechanisms or goroutines were added. This just hooks gracefully into the existing routines. **Limitations:** - Tests against the backend were not written due to the lack of cold storage class support in MinIO. Testing was done manually on Scaleway's S3-compatible object storage. If necessary, we could explore testing with LocalStack or mocks, though this requires further discussion. - Currently, this feature only warms up before restores and repacks (prune/copy), as those are the two main use-cases I came across. Support for other commands may be added in future iterations, as long as affected packs can be calculated in advance. - The feature is gated behind a new alpha `s3-restore` feature flag to make it explicit that the feature is still wet behind the ears. - There is no explicit user notification for ongoing pack restorations. While I think it is not necessary because of the opt-in flag, showing some notice may improve usability (but would probably require major refactoring in the progress bar which I didn't want to start). Another possibility would be to add a flag to send restores requests and fail early. See #3202 * ui: warn user when files are warming up from cold storage * refactor: remove the PacksWarmer struct It's easier to handle multiple handles in the backend directly, and it may open the door to reducing the number of requests made to the backend in the future.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )